Pattern codes used for interactive control of computer applications

ABSTRACT

Methods for determining input to be supplied to a computer program are provided. One of the methods include processing a first video frame having a pattern code before light is applied to the pattern code. The first video frame defines a first characteristic of the pattern code, and the pattern code is defined by at least two tags. The method further includes processing a second video frame having the pattern code when light is applied to the pattern code, such that the second video frame defines a second characteristic of the pattern code. Then, decoding the first characteristic and the second characteristic of the pattern code to produce decoded information. An interactive command is then initiated to the computer program. A type of the interactive command is defined by the decoded information, wherein one of the tags has a reflective surface and one of the tags has a non-reflective surface.

CLAIM OF PRIORITY

This application is a continuation of U.S. application Ser. No.10/842,932, filed on May 10, 2004, and entitled “PATTERN CODES USED FORINTERACTIVE CONTROL OF COMPUTER APPLICATIONS AND VIDEO GAMEAPPLICATIONS,” from which priority is claimed under 35 USC §120, whichis herein incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates generally to computer video games, and moreparticularly to pattern codes which can produce different patternsdepending on when light is applied thereto, and can be decoded to enableinteraction with game play.

2. Description of the Related Art

The video game industry has seen many changes over the years. Ascomputing power has expanded over recent years, developers of videogames have likewise created game software that takes advantage of theseincreases in computing power. To this end, video game developers havebeen coding games that incorporate sophisticated operations andmathematics to produce a very realistic game experience.

Example gaming platforms, may be the Sony Playstation or SonyPlaystation2 (PS2), each of which is sold in the form of a game console.As is well known, the game console is designed to connect to a monitor(usually a television) and enable user interaction through handheldcontrollers. The game console is designed with specialized processinghardware, including a CPU, a graphics synthesizer for processingintensive graphics operations, a vector unit for performing geometrytransformations, and other glue hardware, firmware, and software. Thegame console is further designed with an optical disc tray for receivinggame compact discs for local play through the game console. Onlinegaming is also possible, where a user can interactively play against orwith other users over the Internet.

As game complexity continues to intrigue players, game and hardwaremanufacturers have continued to innovate to enable additionalinteractivity. In reality, however, the way in which users interact witha game has not changed dramatically over the years.

In view of the foregoing, there is a need for methods and systems thatenable more advanced user interactivity with game play.

SUMMARY OF THE INVENTION

Broadly speaking, the present invention fills these needs by providing apattern code that can define different pattern configurations dependingon whether light is applied to the pattern code. The pattern codeconfigurations are then capable of being captured by a video capturedevice, decoded, and depending on the decoded code a specific responseis made by a computer program. The response can be in any form, such asa command, an initiation of action, a selection, a change in status orstate, the unlocking of features, etc.

In one embodiment, a method for determining input to be supplied to acomputer program is disclosed. The method includes processing a firstvideo frame having a pattern code before light is applied to the patterncode. The first video frame defines a first characteristic of thepattern code, and the pattern code is defined by at least two tags. Themethod further includes processing a second video frame having thepattern code when light is applied to the pattern code, such that thesecond video frame defines a second characteristic of the pattern code.Then, decoding the first characteristic and the second characteristic ofthe pattern code to produce decoded information. An interactive commandis then initiated to the computer program. A type of the interactivecommand is defined by the decoded information, wherein one of the tagshas a reflective surface and one of the tags has a non-reflectivesurface.

In another embodiment, an apparatus is disclosed. The apparatus includesa processor for executing program instructions. The program instructionsdefine control instructions for interfacing with a light applicationdevice for applying light and executing instructions for receivingcaptured video from the video capture device. The light applicationdevice is configured to apply light at a first and second tags of apattern code and the control instructions for the light applicationdevice determining when to apply the light. The light being applied atboth of the first and second tags of the pattern code defining a firstpattern configuration when light is applied and a second patternconfiguration that is different than the first pattern configurationwhen the light is not applied. The first and second characteristic eachdefine a code state of the pattern code before and after light isapplied onto the pattern code to define an input to a computer program.

In still another embodiment, a method for interacting with a computergame is disclosed. The method includes presenting a pattern code definedby at least two tags. Applying a light toward the pattern code at aspecific time. Then, the method moves to capturing a first video frameof the pattern code before the light is applied to the pattern code,such that the first video frame defines a first characteristic of thepattern code. A second video frame is then captured for the pattern codewhen the light is applied to the pattern code, such that the secondvideo frame defines a second characteristic of the pattern code. Themethod then enables decoding of the first characteristic and the secondcharacteristic of the pattern code to produce decoded information. Basedon the decoded information, an interactive command is initiated in thecomputer game, and the type of interactive command is defined by thedecoded information that was defined for the computer game.

In a further embodiment, a system for interfacing with a computer gameis disclosed. The system includes the use of a pattern code. The systemis further defined by a light application device, a video capturedevice, and a processor. The processor is capable of executing gameinstructions, executing control instructions for interfacing with thelight application device, and executing instructions for capturing videofrom the video capture device. The light application device isconfigured to apply light at the pattern code and the execution of thecontrol instructions for the light application device determining whento apply the light. The light being applied at the pattern code todefine a first pattern configuration when light is applied and a secondpattern configuration when the light is not applied. The execution ofthe instructions for capturing video is capable of identifying the firstpattern configuration and the second pattern configuration to enable theprocessor to decode the first pattern configuration and the secondpattern configuration to define an interactive instruction for thecomputer game.

In another embodiment, a system for interfacing with a program of acomputer system is provided. Included is a pattern code, a lightapplication device, a video capture device, and a processor. Theprocessor is capable of executing instructions, executing controlinstructions for interfacing with the light application device, andexecuting instructions for capturing video from the video capturedevice. The light application device is configured to apply light at thepattern code and the execution of the control instructions for the lightapplication device is to determine when to apply the light. The light isapplied at the pattern code defining a first pattern configuration whenlight is applied and a second pattern configuration when the light isnot applied. The execution of the instructions for capturing video iscapable of identifying the first pattern configuration and the secondpattern configuration to enable the processor to decode the firstpattern configuration and the second pattern configuration to define aninstruction for the program of the computer system.

In general, the interactive detection of pattern codes is applicable tothe interfacing with any computer program of a any computing device.Thus, any type of code can be embedded in the pattern codes to drive,direct, or instruct action by code executed on a device that isconfigured to do the detection and processing.

Other aspects and advantages of the invention will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings, illustrating by way of example the principles ofthe invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention, together with further advantages thereof, may best beunderstood by reference to the following description taken inconjunction with the accompanying drawings.

FIG. 1A shows a diagram of a game interface system implementing apattern code, in accordance with one embodiment of the presentinvention.

FIG. 1B illustrates an example of a shirt having a number of differentpattern codes of different shapes, sizes, colors, and orientations, inaccordance with one embodiment of the present invention.

FIG. 1C illustrates an example where the pattern code is placed on agame card, in accordance with one embodiment of the present invention.

FIG. 2 illustrates a simplified version of the computing system which isexecuting a game program, in accordance with one embodiment of thepresent invention.

FIGS. 3A-3C illustrate examples of the pattern code, where some tags arereflective and some are not reflective, in accordance with oneembodiment of the present invention.

FIGS. 4A and 4B illustrate examples of how the time intervals can be setfor applying light and capturing pattern code characteristics, inaccordance with one embodiment of the present invention.

FIG. 5A illustrates a flowchart diagram of a method for using a patterncode to control activity in a video game, in accordance with oneembodiment of the present invention.

FIG. 5B illustrates a flowchart diagram of the processing andinter-activity with a pattern code that may be used in a video gameenvironment, in accordance with one embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An invention is disclosed for a pattern code. The pattern code may be inthe form of an article of manufacture. The invention further defines amethod for using the pattern code with a system that implements themethod. The pattern code is defined by one or more tags. Each tagdefines part of the pattern code. Depending on the resulting desiredpattern code, each tag can be made from a material that can reflect orilluminate when light is applied to the material. Thus, each tag of thepattern code can potentially have two visible states. One state whenlight is not applied to the tag of the pattern code and one state whenlight is applied to the tag of the pattern code. The pattern code willthus have one or more of these tags. The pattern code is designed to beplaced in front of a light, and the times of when the light is ON or OFFis controlled. At particular times, therefore, the light will be ON orOFF, and the resulting pattern code will take on a visual characteristicto define a code state for the pattern code.

The visible characteristics are preferably discernable enough so that avideo capture device can determine differences in the individual tags,and can capture the state of each tag when light is applied to the tagand when light is not applied to the tag of the pattern code. When lightis not applied (OFF), there will preferably be some ambient light, suchsome amount of room lighting typically used during general computerinteractivity or computer game play. Once the pattern code is capturedat the particular times of when the light is ON or OFF, the pattern codecan be decoded. Decoding the pattern code will dictate to a game whatcapabilities are enabled for game play. Thus, the pattern code can bedesigned such that different combinations of tags that make up thepattern codes will dictate different capabilities or functionality forinteractive game play.

In the following description, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be apparent, however, to one skilled in the art that the presentinvention may be practiced without some or all of these specificdetails. In other instances, well known process steps have not beendescribed in detail in order not to obscure the present invention.

Keeping the above overview in mind, FIG. 1A shows a diagram of a gameinterface system 100 in accordance with one embodiment of the presentinvention. The game interface system 100 utilizes a computing system 102that couples to a housing 104. The housing includes a video capturedevice 105 and a light application device 110. In an alternativeembodiment, each device can be separately provided in its own individualhousing.

The video capture device 105 may be a video capturing device thatenables frames of images in capture space 105 a to be captured anddigitized before being transferred to the computing system 102. Anexample of the video capture device 105 may be a web cam type videocapture device that captures images and digitizes them into a number offrames as they are transferred to the computing system 102. In anotherembodiment, the video capture device 105 may be an analog-type videocapture device that continuously captures raw video and then transfersit to the computing system 102. In the computing system 102, the rawvideo could then be digitized into frames.

The light application device 110 is preferably a light-emitting devicethat can emit different types of light. The light can be in the form ofa colored light, such as those produced by light-emitting diodes (LEDs).As is well known, LEDs are capable of producing a bright enough lightthat could illuminate objects sitting or placed in front of the LED. Anexample distance for the object, having the pattern code, to be in frontof the applied light may be as close as 1 cm and as far as 20 meters. Amore normal middle distance range may be between 50 cm and about 5meters. In a specific example, the light application device 110 willemit a red light when activated. In another example, the light can takeon any color, or can simply be non-colored to the naked eye (e.g., suchas the light produced by a flash light).

As shown, the housing 104 for the video capture device 105 and the lightapplication device 110 may sit on top of a display 106. The housing 104is shown coupled by a wire to the computing system 102. The computingsystem 102 may be any computer device (i.e., having a processor andmemory) that is capable of executing code and interfacing with the videocapture device 105 and the light application device 110. The computingsystem 102 may be local to the housing 104 or can be remotely coupled toanother computing device over a network. In a specific example, thecomputing system 102 may be a game console such as the Sony Playstation2. The computing system 102 would then be capable of playing a game thatallows a player 112 to interface with graphics of the game.

For a more detailed description of a technique for interfacing with agame by detecting movement of a player, reference may be made to UnitedKingdom Applications GB 0304024.3 (PCT/GB2004/000693) and GB 0304022.7(PCT/GB2004/000703), each filed on Feb. 21, 2003, and each of which ishereby incorporated by reference.

The video capture device 105 is designed to identify movement 114 of theplayer 112 to enable interaction with features of a game. In theillustrated example, the player 112 is shown as a translucent image 112′on the display 106. The player 112 can then move his hand (or any partof his body or objects held by the player) as shown by movement 113 tocause interaction with objects of the game.

The movement 114 is shown as movement by the translucent image 112′ toproduce an active region 113′. The active region 113′ is the region thatwas identified to have motion, in response to processing motiondifferences between frames. The active region 113′ will then cause theobject defined as a flag 124 to move. Thus, the player 112 is allowed tointeract with the flag 124 held by a game character 122 (or otheraspects of the game features or control features).

In accordance with one embodiment, the player 112 is shown wearing ashirt 121 that includes a pattern code 120. The pattern code 120 isdefined by one or more tag codes, and each tag code may be able tochange state when light is applied to the individual tags that make upthe pattern code 120. The pattern code 120 shown in FIG. 1A isillustrated by six individual tags. Each individual tag may be definedby a different type of fabric or material. The material or fabric isdesigned to be placed on, sewed on, or applied to the shirt 121 at anylocation, so long as the pattern code 120 is capable of beingilluminated by applied light 110 a that is emitted by the lightapplication device 110.

The pattern code 120 should also be placed on the player 112 in alocation that would be captured and can be identified by the videocapture device 105. Accordingly, the pattern code 120 is configured tohave an identifiable pattern that can be captured by the video capturedevice 105, decoded, and understood to be a particular patternconfiguration. When the light application device 110 is turned ON, anapplied light 110 a is capable of being illuminated on the pattern code120. When the pattern code 120 is illuminated with the applied light 110a, the pattern code 120 will take on a different pattern codecharacteristic. Thus, the pattern code 120 is designed to have a firstpattern code configuration when the light application device 110 is OFF,and second pattern configuration when the light application device 110is ON. As noted above, when the light is OFF, there will preferably besome ambient light present. The ambient light can be any light that istypically present during game play. Such ambient light may be roomlight, or can be the light that is given off by the display.

The light application device 110 is designed to be controlled to the ONand OFF position by code that is executed on the computing system 102.As will be described in greater detail below, the computing system willexecute code that determines when the light application device 110 willturn ON and OFF. The duration of which the light will be turned ON andOFF is also controllable to enable the video capture device to capturethe resulting pattern code characteristic that results when light isapplied or not applied to the pattern code 120. The video capture device105 is thus designed to capture the different visible conditions of thepattern code 120 and decoding logic is executed by the computer system102, to determine what the pattern code 120 signifies.

In a preferred embodiment, the pattern code 120 is associated with ashirt 121 that is purchased by the player 112 for use with a specificgame. When the user purchases shirt 121 with the specific pattern 120 onthe shirt, the user may be provided with special powers when playing thevideo game associated with the shirt 121. For purposes of example, whenthe player 112 is wearing shirt 121 with pattern code 120, the gamewould allow the player 112 to have special powers to, for example, turnthe game character 122 into Superman.

If the player 112 were not using the shirt 121 with the pattern code120, the game character 122 would not be Superman, and would not havespecial powers. In general terms, the pattern code 120 is used by theplayer 112 to enable a special characteristic of the game when the useris interacting with the game through the video capture device 105, andthe light application device 110 is used to enable decoding of what thepattern 120 symbolizes, and would thus enable the player to activate afeature of a video game. In one embodiment, the pattern 120 may be usedto “unlock” an aspect of a video game (e.g., a car, a track, etc.),which would otherwise be unavailable to the user. The pattern 120 cantake on any number of forms, and in one embodiment, the form can be aholistic graphic, where part of the graphic is reflective and the otherpart is not reflective.

FIG. 1B illustrates an example of the shirt 121 having a number ofdifferent pattern codes 120 having different shapes, sizes, andorientations. As mentioned above, each tag piece of the pattern codes120 would define a particular pattern. Each tag of the pattern code 120is capable of having two states. The first state would be theidentifiable state when no light is being applied to the tag, and asecond state when light is being applied to the tag.

Depending on the material, characteristic, or color of the particulartag, the tag will have a particular identifiable state when the light isOFF, and another particular identifiable state when the light is ON andapplied to the individual tag. The pattern codes can be sewed onto ashirt 112, for example, or can be stuck on to a shirt to enable the userto interact with a particular game. In another embodiment, the shirt 121could have the individual tags that make up a pattern code 120 blendedinto the fabric of the shirt 121. In either case, the individual tagsare designed to have two states, one when the light is OFF and notapplied against the tag, and one when the light is ON and the light isapplied to the tag. If the particular tag is designed to reflect whenlight is applied to the tag, the reflected light would pose a differentpattern to the video capture device 105.

In a like manner, if the light is applied to the particular tag of thepattern code 120 and no reflection occurs, then the video capture device105 will capture that information and communicate it to the computingsystem 102. In one embodiment, the pattern codes 120 are designed tohave a pattern that is not readily discernible to the naked eye withoutthe use of special lighting. Thus, the pattern that is portrayed to theuser by the pattern codes 120 when the light is OFF, will be completelydifferent than the pattern that results when light is applied to thepattern codes 120. In this manner, a user is not able to easilycounterfeit the pattern code 120 by simply coloring in a card withdifferent shades of colors that are similar to those that are providedon the shirt 121. This is because when light is applied to the patterncodes 120, the resulting pattern will depend on which individual tagsare meant to be reflective. In one example, such reflecting materialsare called “retro-reflective”.

Referring again to FIG. 1B, the pattern codes 120, once decoded usingthe ON and OFF states of the applied light 110 a, will provide the userinteracting with the game special powers, abilities, interfacealternatives, or controlling parameters that are not normally availablewithout displaying the pattern codes 120 to the video game being played.In this example, the game character 122 is shown to be Superman, whileif the pattern codes 120 represented different information, the gamecharacter 122 might resemble a different character or have differentpowers. It should be understood that the resulting action in the actualgame will vary depending on the circumstances of the game and what eachparticular pattern is meant to do with reference to game play.

FIG. 1C illustrates an example where the pattern code 120 is placed on agame card 140. Instead of wearing the pattern code 120, the pattern code120 may be provided on a game card 140. A user may then collect or usedifferent game cards 140 to enable different powers or differentcapabilities for interacting with a game 132. In this example, the useris provided with special powers illustrated by icon 134.

The special powers for playing the game 132 may be illustrated by icon134 and may provide the user with additional interactive icons 130. Theadditional interactive icons 130 may not have been provided to the user,but for the fact that the game card 140 was used with the proper patterncode 120. Following this example, the movement 114 of the player 112 canbe used to generate an active region 114′. The active region 114′ mayact to select the interactive icon “E”. By selecting the icon “E”, theuser may cause special interaction with the game 132 that may not havebeen provided, but for the fact that the user was using game card 140with the pattern code 120.

As mentioned above, the pattern code 120 is configured to have a patterndistribution with individual tags. Each individual tag may be eitherreflective or not reflected, and will have a discernible characteristicwhen the light is applied to the tag and when light is not applied tothe tag. By using the pattern code produced by the two situations thatwhen light is applied to the tag and when light is not applied to thetag, the capture device will take in that information in different videoframes. The computing system 102 using code operated to detect thedifferent patterns, will compute what abilities are provided by the gamecard 140.

FIG. 2 illustrates a simplified version of the computing system 102which includes a processor 151 and is executing a game program 150. Thegame program 150 is configured to include pattern code detection code152. As shown, the computing system 102 is coupled to each of the videocapture device 105 and the light application device 110. The videocapture device 105 is coupled to the computing system 102 such thatcapture information is communicated to the computing system in one ofdigital form or analog form, depending on the particular system.

For instance, if the video capture device 105 is capable of digitizingthe captured video, the captured information is communicated in digitalform to the computing system 102. The digital form of the captured videomay be in the form of a plurality of captured frames of video. The lightapplication device 110 is coupled to the computing system 102 to providelight control. Light control will dictate when the light applicationdevice 110 will turn ON and OFF. The object having pattern code 120 isshown having tag codes 120 a. One of the tag 120 a is identified to bewhite “W”, and the other tag 120 a is identified to be black “B”.

Thus, the visual representation of the tag codes 120 a of the patterncode 120 will have a black and white representation when the light isOFF and only ambient light is present. Although the colors black andwhite are being used for discussion purposes, it should be understoodthat the colors can be varying colors of white and varying colors ofblack, and also the colors do not need to be black or white, but can beany color that can provide a contrast between adjacent colors.

In this example, the white tag code 120 a is made from a material thatis reflective. As mentioned above, the reflective material can be a typeof cloth, or plastic-type material that is capable of reflecting lightwhen light is applied to the surface of the material. These materialsmay reflect light in many directions when light is applied to thesurface. However, when the light is OFF, the tag code 120 a having thewhite surface, may look very similar to the surface of the material thatis shown to be black. In display 106, the computing device 120 maygenerate a first frame (captured frame (n)), which is illustrated tohave a captured image of the pattern code 120. The first frame wascaptured by the video capture device 105 when the light applicationdevice 110 was OFF. Thus, the video capture device 105 would havecaptured the frame where the top tag is light in color and the bottomtag is dark in color. In a next captured frame (n+1), the pattern code120 will have a different state. The different state in the capturedframe for the pattern code 120 is the result of having had the lightapplication device 110 ON.

When the light application device 110 was ON, the top tag becameilluminated to a bright state thus reflecting light. However, the bottomtag did not have the reflective material built into the tag portion, andtherefore, the detected pattern code 120 will have the top tag brighterthan the bottom tag. Consequently, the computing system 102 will capturethe information from each of the captured frames and process the tagcodes 120 a of the pattern code 120 during each of the times when thelight was OFF and the light was ON.

Although shown as two simple tags for illustration purposes, the patterncode 120 can have any number of tag codes 120 a, and form differentshapes, patterns and arrays. In addition, the computing system 102 maybe programmed to capture any number of frames and compare the resultingpattern codes 120 captured as a result of the controlled application bythe light application device 110, to the pattern code 120.

FIG. 3A illustrates an example where the pattern code 120 has white andblack materials that portray a pattern code A when the light applicationdevice 110 is OFF. The video capture device 105 will capture the imageof the pattern code 120 to produce the pattern code A. As illustrated,the pattern code A will have two top tags 120 a that are white, and twobottom tags 120 a that are white, and a middle row of black tags 120 a.In accordance with one embodiment, any number of the tags can be madefrom a reflective material. In this example, the top left white tag isreflective, the left black tag is reflective, and the bottom right whitetag is reflective.

However, to the casual observer of the pattern code 120, the observerwill only see the pattern code having the two black tags 120 a in themiddle row when the light is OFF. When the light is ON, as shown in FIG.3B, the tags 120 a that have the reflective material will beilluminated, thus producing a pattern code B. The pattern code B willthus be captured by the video capture device 105. From this example, itis evident that the pattern code A of FIG. 3A is different than thepattern code B of FIG. 3B. Thus, no matter what the color or shade ofthe individual tags 120 a are, it is possible to make the differentcolors reflective or non-reflective, depending on the pattern that isdesired when the light is ON as shown in FIG. 3B. FIG. 3C illustratesthe example where different tags 120 a are made to be reflective andnon-reflective. Depending on where the reflective tags 120 a (R) areplaced in the pattern code 120, the reflective tags will cause adifferent pattern to be captured by the video capture device 104 whenthe light is ON.

In an aspect of the invention, it should be noted that the visual imagewhen the light is ON can be subtracted from when the light is OFF, tosee the difference between the two. By performing the subtraction, it ispossible to make decoding of the images more robust. In this aspect ofthe invention, the robustness is believed due to the fact that it iseasier to see how light changes, rather than comparing the image to anabsolute image of what is expected (which could vary due to lightingconditions, etc). Thus, the change due to the light should be quiteinsensitive to other visual disturbances.

FIG. 4A illustrates a table of the light application status andcomparing the detected pattern code for pattern codes being analyzed,and each captured frame. In frame (n−3), the light application status isON, and the detected pattern code is B. In frame (n−2), the lightapplication status is ON, and the detected pattern code is B. In frame(n−1), the light application status is OFF, and the detected patterncode is A. In frame (n), the light application status is ON, and thedetected pattern code is B. And in the frame (n+1), the lightapplication status is OFF, and the detected pattern code is A. Thus, thecomputing system 102 can determine when the light application statuswill be ON and OFF, and the frequency at which it is turned ON and OFF,or the number of frames it is left ON or OFF. In this example, fiveframes were analyzed to determine when the light application status wasON and OFF, and the resulting detected pattern code is produced. Thedetected pattern code for the five frames that were captured and decodedwould be “B B A B A”.

FIG. 4B illustrates a special case where color light is applied by thelight application device. The color light may be in addition to a plainwhite light (e.g., similar to light emitted by a flash light). Inanother embodiment, the color light may be the light applied by a redLED, or a different colored lens that would apply a different color tothe pattern code. In FIG. 4B, during the light application status forframes (n−3) and (n), a color light is applied to the pattern code. Inthis situation, the color light is different than the light that wasapplied when the light application status was ON in FIG. 4A.

For instance, the light may have been a white light in FIG. 4A, or thelight may have been a red light in FIG. 4A. But in FIG. 4B, the colorlight is a light other than the light that was being applied in FIG. 4A.In such a scenario, the detected pattern code would be slightlydifferent (i.e., B′) for frames (n−3) and (n), as compared to frame(n−2). Thus, it is possible to not only detect whether light is ON, butalso detect different shades of color that can be reflected andcontrasted on the pattern code that is being analyzed. Thus, a tag cantake on more that two states. The number of states that a tag can takeon is dependent on whether the light is ON or OFF, but also on thecolors used. Still further, an embodiment of the present invention caninclude having multiple light sources, and each light source can producea different color shade. Each light source can thus be turned ON or OFFin accordance with its own coding and decoding scheme.

FIG. 5A illustrates a flowchart diagram of a method for using a patterncode to control activity in a video game. The method begins at operation202 where a request is made to detect pattern codes from an objectplaced in front of a video capture device. The request may be made bythe software that is executing in the computing system, such as thesoftware that is part of a video game. In operation 204, the lightapplication device would receive a command from the software to apply alight on the object for one or more ON and OFF intervals. In anotherembodiment, the intervals may be multiple ON's followed by multiplesOFF's or any combination of ON/OFF, to define specific ON/OFF intervals.

The method now moves to operation 206 where pattern codes are capturedfrom the object during each of the ON and OFF intervals. The capturedpattern codes are then detected in operation 208 for each of thecaptured ON and OFF intervals. Based on the decoded information obtainedfrom the pattern codes with respect to the ON and OFF intervals, thesoftware of the video game can cause an initiation in game control inresponse to the decoded captured pattern codes. The initiation of gamecontrol may be in the form of providing special interactive features tothe game, or enabling special features of the game, or unlockingadvanced features that can only be unlocked when the user provides thepattern code that can be detected by the capture device.

FIG. 5B illustrates a flowchart diagram 220 of the processing andinter-activity with a pattern code that may be used in a video gameenvironment. In operation 222, a computing device is provided forenabling a game play. The computing device can be in the form of ageneral computer, or a specialized computing system such as a gameconsole (e.g., the Sony Playstation 2). The method then moves tooperation 224 where a video frame capture device is provided. The videoframe capture device can be connected to the computing device to enablethe captured frames to be processed by the computing device.

The method then moves to operation 226 where a user is allowed tointeract with game play by making movements in front of the videocapture device. Examples of the movement have been described withreference to FIGS. 1A and 1C. In operation 228, pattern codes aredetected from an object that is associated with the user. The objectshould be placed in front of the video capture device to enable it todetect the pattern codes. The times at which the pattern codes aredetected are set by the computer code that would be running andassociated with the game play. At particular times which are set by thegame processing logic, a command is sent to the light application deviceto apply light to the object for one or more ON/OFF intervals of thelight in operation 230.

The video frame capture device would then capture the pattern codes fromthe object during each of the ON and OFF intervals in operation 232. Thepattern codes are then decoded for each of the captured ON and OFFintervals to determine attributes associated with the captured patterncodes in operation 234. Depending upon the decoded pattern code, thevideo game will either provide or not provide special powers or specialinteractivity to the user during game play. In operation 236, gamecontrol may be initiated in response to the determined attributesassociated with the captured pattern codes. Initiating game controlshould be broadly understood to be any kind of control or attribute thatis enabled or disabled in view of the pattern code that was detected andused by the user during game play.

As described herein, embodiments of the invention have been defined withreference to a game program. However, it should be understood that thepattern codes can be used in any number of applications outside of gameprograms. The pattern codes, for instance, can be used in situationswhere hard-to-fake pattern encoding and decoding is needed. In someinstances, it can be used to track merchandise, track vehicles, trackobjects in transit, track inventory of goods, track people that enter arestricted place or have authority to enter certain area, or to identifypersons that have access to certain programs or services. In oneexample, a user having the pattern codes, e.g., either wearing ordisplaying, can access services provided by a walk-up kiosk or the like.Accordingly, the coding and decoding of pattern codes should be viewedbroadly, and should not be restricted to game programs.

In addition, the pattern codes can be encoded with data, and thedecoding can be configured to read and understand the data stored on thepattern codes. The reading of the pattern codes, as noted above, canoccur during times when the light is ON, OFF, or when the light is ON indifferent colors. The pattern codes can also store a serial number,data, or any type of identification data. Such serial number, data, oridentification information can then be used by a computer to initiate orexecute task or program, and generate a result.

Still further, in addition to being a hard to fake pattern code, thepattern code is also a reliable way to encode more data into the samearea. In the special case of a binary situation, as used in twodimensional matrix-style barcodes, a given area could encode 2 bitsrather than just 1 bit. In effect, the pattern codes would present 2independent barcodes, and could be decoded exactly in that way, using atwo dimensional matrix-style barcode decoder methodology, but with anextension. For instance, a process could involve performing atraditional image decode with the light OFF, which generates firstdecoded data. A second process could include decoding with the lightOFF, and then decoding with the light ON, and then taking the different.The difference would then represent second decoded data. As noted above,this technique can provide a more robust decoding methodology, as lightdisturbances would be less likely to affect the result obtained by thesubtraction process.

Still further, a pattern code can take on any particular shape. Thespaces illustrated here are only exemplary. For instance, the patterncodes that are arranged in a row and column array do not need to besymmetric, and each tag of the pattern code can be made larger orsmaller depending on the media onto which it is to be applied. Also, thetags can be joined to form geometric shapes, such that the pattern codeincludes only different shapes, or the shapes can be mixed withindividual square shapes.

It should be appreciated that the embodiments described herein may alsoapply to on-line gaming applications. That is, the embodiments describedabove may occur at a server that sends a video signal to multiple usersover a distributed network, such as the Internet, to enable players atremote noisy locations to communicate with each other. It should befurther appreciated that the embodiments described herein may beimplemented through either a hardware or a software implementation. Thatis, the functional descriptions discussed above may be synthesized todefine a microchip having logic configured to perform the functionaltasks for each of the modules associated with the noise cancellationscheme.

With the above embodiments in mind, it should be understood that theinvention may employ various computer-implemented operations involvingdata stored in computer systems. These operations include operationsrequiring physical manipulation of physical quantities. Usually, thoughnot necessarily, these quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated. Further, the manipulationsperformed are often referred to in terms, such as producing,identifying, determining, or comparing.

The above described invention may be practiced with other computersystem configurations including hand-held devices, microprocessorsystems, microprocessor-based or programmable consumer electronics,minicomputers, mainframe computers and the like. The invention may alsobe practiced in distributing computing environments where tasks areperformed by remote processing devices that are linked through acommunications network.

The invention can also be embodied as computer readable code on acomputer readable medium. The computer readable medium is any datastorage device that can store data which can be thereafter read by acomputer system, including an electromagnetic wave carrier. Examples ofthe computer readable medium include hard drives, network attachedstorage (NAS), read-only memory, random-access memory, CD-ROMs, CD-Rs,CD-RWs, magnetic tapes, and other optical and non-optical data storagedevices. The computer readable medium can also be distributed over anetwork coupled computer system so that the computer readable code isstored and executed in a distributed fashion.

Although the foregoing invention has been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications may be practiced within the scope of theappended claims. Accordingly, the present embodiments are to beconsidered as illustrative and not restrictive, and the invention is notto be limited to the details given herein, but may be modified withinthe scope and equivalents of the appended claims.

1. A method for determining input to be supplied to a computer program,comprising: processing a first video frame having a pattern code beforelight is applied to the pattern code, the first video frame defining afirst characteristic of the pattern code, the pattern code defined by atleast two tags; processing a second video frame having the pattern codewhen light is applied to the pattern code, the second video framedefining a second characteristic of the pattern code; decoding the firstcharacteristic and the second characteristic of the pattern code toproduce decoded information; and initiating an interactive command tothe computer program, a type of the interactive command defined by thedecoded information; wherein one of the tags has a reflective surfaceand one of the tags has a non-reflective surface.
 2. The method claim 1,wherein the second characteristic of the pattern code has moreillumination coming from the reflective surface while less coming fromthe non-reflective surface.
 3. The method of claim 1, wherein the firstcharacteristic of the pattern code has a color that is darker in thenon-reflective surface and lighter in the reflective surface.
 4. Themethod of claim 1, wherein each of the two tags is capable of having aspecific color.
 5. The method of claim 1, wherein the second patternconfiguration that is define when the light is not applied to thepattern code is visually defined by characteristics of each portion ofthe tags of the pattern code.
 6. The method of claim 1, wherein thecomputer program is one of an application or a computer game.
 7. Themethod of claim 1, further comprising, arranging tags in a row andcolumn array that is either symmetric or non-symmetric.
 8. The method ofclaim 1, further comprising, arranging tags in a joined configuration todefine geometric shapes.
 9. The method of claim 1, wherein the type ofthe interactive command defined by the decoded information includestracking one of merchandise, vehicles, objects in transit, inventor ofgoods, people entering a place to verify authority, people that haveaccess to programs or services.
 10. The method of claim 1, wherein thetags are associated with a user by either wearing of the tags ordisplaying of the tags, or an object having the tags associatedtherewith.
 11. The method of claim 1, wherein the type of theinteractive command defined by the decoded information triggers anaction by a video game, the action being one of a special power orspecial interactivity during play of the video game.
 12. The method ofclaim 1, wherein the type of the interactive command defined by thedecoded information causes access to an attribute during play of a videogame.
 13. The method of claim 1, further comprising; determining a colorof the light that is applied, the color including any one of white lightor a color variation that is different than white, where a differentcolor acts to cause the type of interactive command defined by thedecoded information to change.
 14. The method of claim 1, wherein thereflective surface is one of a cloth material or a plastic-type materialthat is capable of reflecting the light.
 15. The method of claim 12,wherein the access is one of an unlock of an aspect of the video game.16. The method of claim 1, wherein the at least two tags form a holisticgraphic, where part of the holistic graphic is reflective and anotherpart is not reflective.
 17. An apparatus, comprising: a processor forexecuting program instructions, the program instructions definingcontrol instructions for interfacing with a light application device forapplying light, and executing instructions for receiving captured videofrom the video capture device, the light application device beingconfigured to apply light at a first and second tags of a pattern codeand the control instructions for the light application devicedetermining when to apply the light, the light being applied at both ofthe first and second tags of the pattern code defining a first patternconfiguration when light is applied and a second pattern configurationthat is different than the first pattern configuration when the light isnot applied, wherein the first and second characteristic each define acode state of the pattern code before and after light is applied ontothe pattern code to define an input to a computer program.
 18. TheApparatus of claim 17, wherein one of the tags has a reflective surfaceand one of the tags has a non-reflective surface.
 19. The Apparatus ofclaim 17, wherein the first pattern configuration has more illuminationdefined from the reflective surface and less illumination from thenon-reflective surface.
 20. The Apparatus of claim 17, wherein thesecond pattern configuration that is define when the light is notapplied to the pattern code is visually defined by characteristics ofeach portion of the tags of the pattern code.
 21. The Apparatus of claim17, wherein the characteristics of each tag is defined by color.
 22. Acomputer program embodied on a computer readable medium for determininginput to be supplied to the computer program, the computer programcomprising: program instructions for processing a first video framehaving a pattern code before light is applied to the pattern code, thefirst video frame defining a first characteristic of the pattern code,the pattern code defined by at least two tags; program instructions forprocessing a second video frame having the pattern code when light isapplied to the pattern code, the second video frame defining a secondcharacteristic of the pattern code; program instructions for decodingthe first characteristic and the second characteristic of the patterncode to produce decoded information; and program instructions forinitiating an interactive command to the computer program, a type of theinteractive command defined by the decoded information; wherein one ofthe tags has a reflective surface and one of the tags has anon-reflective surface.
 23. The computer program of claim 22, whereinthe second characteristic of the pattern code has more illuminationcoming from the reflective surface while less coming from thenon-reflective surface.
 24. The computer program of claim 22, whereinthe first characteristic of the pattern code has a color that is darkerin the non-reflective surface and lighter in the reflective surface. 25.The computer program of claim 22, wherein each of the two tags iscapable of having a specific color.
 26. The computer program of claim22, wherein the second pattern configuration that is define when thelight is not applied to the pattern code is visually defined bycharacteristics of each portion of the tags of the pattern code.
 27. Thecomputer program of claim 22, wherein the computer program is one of anapplication or a computer game.
 28. A method for analyzing light data,comprising: identifying in a first video frame a pattern code beforelight is applied to the pattern code, the first video frame defining afirst characteristic of the pattern code, the pattern code defined by atleast two tags; identifying in a second video frame the pattern codewhen light is applied to the pattern code, the second video framedefining a second characteristic of the pattern code; decoding the firstcharacteristic and the second characteristic of the pattern code toproduce decoded information; and initiating an interactive command to acomputer program, a type of the interactive command defined by thedecoded information, wherein one of the tags has a reflective surfaceand one of the tags has a non-reflective surface.